Plot individual images on the cluster¶

This notebook enables up to view raster outputs (.tif and .nc) on the cluster without needing to transfer files

Imports¶

Basic Parameters: 
 PrintDate = 2023-05-28 
 brdf_dir = /home/sandbox-cel/paraguay_lc/stac/grid/004057/brdf 
 data source = stac 
 gridCell = 4057 
 index_dir = /home/downspout-cel/paraguay_lc/stac/grids/004057/brdf_ts/ms/evi2 
 out_dir = /home/klwalker/data/tmp
Plotting Parameters: 
 Viewband = nir 
 imageType = Sentinel 
 plotYr = 2021 
 plotDay = 100
closest downloaded Sentinel image is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/sentinel2/S2A_21JYM_20210410_0_L2A.tif with file size 48855339
closest Sentinel brdf is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/brdf/L3B_S2A_brdf21JYM0L2A_20210410_20230217_C01_coreg.nc with file size 32449684
closest downloaded Landsat image is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/landsat/LC08_L2SP_224078_20210403_02_T1.tif with file size 108342256
closest downloaded Landsat5 image is:
     there are no Landsat5 images for target year 2021
closest downloaded Landsat7 image is:
     there are no Landsat7 images for target year 2021
closest downloaded Landsat8 image is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/landsat/LC08_L2SP_224078_20210403_02_T1.tif with file size 108342256
closest downloaded Landsat9 image is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/landsat/LC09_L2SP_224078_20211103_02_T1.tif with file size 108324985
closest Landsat brdf is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/brdf/L3A_LC08_brdf22407802T1_20210403_20230217_C01.nc with file size 33322059
closest smoothed evi2 index image is:
/home/downspout-cel/paraguay_lc/stac/grids/004057/brdf_ts/ms/evi2/2021100.tif
/home/downspout-cel/paraguay_lc/stac/grids/004057/brdf_ts/ms/evi2/2021100.tif with file size 8858221
plotting nir band of image /home/sandbox-cel/paraguay_lc/stac/grid/004057/brdf/L3B_S2A_brdf21JYM0L2A_20210410_20230217_C01_coreg.nc
this is in .nc format
epsg:8858
['blue', 'green', 'nir', 'red', 'swir1', 'swir2']
Coord range is: y: -3279710.0--3259500.0. x: 3198730.0-3218940.0
The no data value is: nan
plotting nir band of image /home/sandbox-cel/paraguay_lc/stac/grid/004057/sentinel2/S2A_21JYM_20210410_0_L2A.tif
this is in .tif format
The no data value is: 32768.0
min and max nir are 327.0 - 32768.0
plotting nir band of image /home/sandbox-cel/paraguay_lc/stac/grid/004057/landsat/LC08_L2SP_224078_20210403_02_T1.tif
this is in .tif format
The no data value is: 32768.0
min and max nir are nan - nan

Plot 3-band image¶

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

Compare original, downloaded and brdf image:¶

Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).

File checks -- more complete in other notebooks¶

/home/klwalker/Jupyter/LUCinSA_helpers/LUCinSA_helpers/notebooks/../LUCinSA_helpers/file_checks.py:86: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_lan['date'] = df_lan.index.map(lambda x: int(x.split('_')[3][:8]))
/home/klwalker/Jupyter/LUCinSA_helpers/LUCinSA_helpers/notebooks/../LUCinSA_helpers/file_checks.py:88: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  df_sen['date'] = df_sen.index.map(lambda x: int(x.split('_')[2][:8]))
id dl beforeDB brdf_id brdf_error brdf bandpass coreg shift_x shift_y coreg_error dldate numpix redownload error skip skip_reason sensor date
LC09_L2SP_224078_20211223_02_T1 /home/sandbox-cel/paraguay_lc/stac/grid/004057... True L3A_LC09_brdf22407802T1_20211223_20230217_C01.nc NaN True False False NaN NaN NaN NaN NaN NaN NaN NaN NaN lc09 2021-12-23
LC08_L2SP_224078_20211028_02_T1 /home/sandbox-cel/paraguay_lc/stac/grid/004057... True L3A_LC08_brdf22407802T1_20211028_20230217_C01.nc NaN True False False NaN NaN NaN NaN NaN NaN NaN NaN NaN lc08 2021-10-28
LC08_L2SP_224078_20210926_02_T1 /home/sandbox-cel/paraguay_lc/stac/grid/004057... True L3A_LC08_brdf22407802T1_20210926_20230217_C01.nc NaN True False False NaN NaN NaN NaN NaN NaN NaN NaN NaN lc08 2021-09-26
LC08_L2SP_224078_20210129_02_T2 /home/sandbox-cel/paraguay_lc/stac/grid/004057... True L3A_LC08_brdf22407802T2_20210129_20230217_C01.nc NaN True False False NaN NaN NaN NaN NaN NaN NaN NaN NaN lc08 2021-01-29
LC08_L2SP_224078_20210113_02_T1 /home/sandbox-cel/paraguay_lc/stac/grid/004057... True L3A_LC08_brdf22407802T1_20210113_20230217_C01.nc NaN True False False NaN NaN NaN NaN NaN NaN NaN NaN NaN lc08 2021-01-13
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/core/indexes/base.py:3802, in Index.get_loc(self, key, method, tolerance)
   3801 try:
-> 3802     return self._engine.get_loc(casted_key)
   3803 except KeyError as err:

File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/_libs/index.pyx:138, in pandas._libs.index.IndexEngine.get_loc()

File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/_libs/index.pyx:165, in pandas._libs.index.IndexEngine.get_loc()

File pandas/_libs/hashtable_class_helper.pxi:5745, in pandas._libs.hashtable.PyObjectHashTable.get_item()

File pandas/_libs/hashtable_class_helper.pxi:5753, in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'file_path'

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
Cell In[39], line 5
      2 end = datetime.datetime.strptime('{}-12-31'.format(single_plot_params['plot_yr']),'%Y-%m-%d').date()
      4 dfslice = processing_db[start:end]
----> 5 brdf_file_list = dfslice['file_path'].tolist()
      6 dl_file_list = dfslice['orig_file_path'].tolist()
      7 #print('There are {} files in file list'.format(len(file_list)))

File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/core/frame.py:3807, in DataFrame.__getitem__(self, key)
   3805 if self.columns.nlevels > 1:
   3806     return self._getitem_multilevel(key)
-> 3807 indexer = self.columns.get_loc(key)
   3808 if is_integer(indexer):
   3809     indexer = [indexer]

File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/core/indexes/base.py:3804, in Index.get_loc(self, key, method, tolerance)
   3802     return self._engine.get_loc(casted_key)
   3803 except KeyError as err:
-> 3804     raise KeyError(key) from err
   3805 except TypeError:
   3806     # If we have a listlike key, _check_indexing_error will raise
   3807     #  InvalidIndexError. Otherwise we fall through and re-raise
   3808     #  the TypeError.
   3809     self._check_indexing_error(key)

KeyError: 'file_path'

To save an html copy of this notebook with all outputs:¶